External Memory Parallel Sorting by Sampling

This paper introduces an external memory parallel sorting algorithm in a multiprocessor architecture. The overall goal is to choose p − 1 partitioning elements so that the final p sorted files, one per processor, are of roughly equal size. It first determines a sample of splitters by either regular sampling or random sampling techniques. Then each data file at each processor is separated according to final splitters and sublists are redistributed to appropriate processors. Finally each processor sorts incoming records into runs and merges sorted runs into a fully sorted file. We implemented our algorithm using C and MPI package and tested its performance on both a cluster of SUN Solaries workstations and a Linux cluster CGM1. The result indicates that regular sampling provides better performance than random sampling does.